Method and system for dynamic adjustment of a model

ABSTRACT

A method and system for manipulating a 3D model during treatment planning and automatically adjusting the 3D model based on a localized area of the 3D model proximate to a location of said manipulation. The 3D model may be for example, a 3D model of a tooth or teeth or dental anatomy.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation application that claims the benefitof U.S. patent application Ser. No. 16/023,671 filed on Jun. 29, 2018which is herein incorporated by reference for all purposes.

FIELD OF THE INVENTION

The present application relates generally to a method for dynamicallyadjusting model views and, more particularly, to a method and system fordynamically adjusting a view and/or an orientation of a model based on alocalized area of the model during treatment planning.

BACKGROUND OF THE INVENTION

Various options exist available for treating dental defects such as bydesigning restorations or orthodontic aligners. One option is the manualapplication of filling material in the mouth wherein dentist removestooth decay and fills a created hole with a filling material. Anotheroption for larger defects includes taking a physical impression by thedentist which may be sent to a dental laboratory to create a plastermodel. By taking account of the opposing teeth and, if appropriate, thejaw movements in the form of articulators, it may be possible to produceappropriate tooth restoration or dental prosthetic items such as inlays,onlays, partial crowns, crowns, bridges, telescope crowns, partialprostheses, etc.

However these manual processes are time consuming and expensive. CAD/CAMtechnology may be an alternative to conventional production methods, inwhich the dental restorations and dental prosthetic items may beproduced with the aid of a computer. Three-dimensional (3D) data may beacquired with hardware such as a dental scanner. A computer-aided design(CAD) data set of a tooth restoration may then be virtually designed orcomputed using the scanned 3D data and a display. The finished CAD dataset may then be machined in a computer-controlled milling or grindingmachine (e.g. CNC) or rapid prototyping systems. However designing thevirtual restoration is time consuming as the user has to manually changeview angles of the restoration during treatment planning in order tohave a clear view of a part of the tooth surface being designed.

In one approach, a user may position a pointer over a digital imageshown on a screen of a display to cause a magnified view of a portion ofthe digital image to appear in a box in a fixed location on the screen.This approach may be unfavorable in that the box may cover part of thedigital image and the user may need to simultaneously focus on twodifferent areas which may strain the eyes. Moreover such magnified viewshave only been known to be generated for 2D images.

Further, some text editors may allow a user to change the level ofmagnification at which a document is being displayed. This approach maybe unfavorable because the change of magnification occursinstantaneously, and as a result, the user may become disoriented as towhat he or she is viewing after a change in magnification. Further, theentire document may be displayed at the same magnification level, and soit is not possible to view the “big picture” while inspecting a portionof the document at a higher level of magnification. Further, a user maywant to modify the document in ways other than just magnification.

U.S. Pat. No. 9,372,590 discloses a magnifier movement system wherein adrag control element is selected for moving a magnifier over a displayarea.

During dental treatment, a tooth may be scanned prior to preparation,together with the adjacent teeth, and a corresponding 3D model may thenbe computed.

U.S. Pat. No. 7,705,858 discloses a method and apparatus for displayingdigital images on a display wherein a pile of images that include aplurality of images arranged in a first arrangement in which at leastone image in the pile overlaps with at least one other image in the pilemay be displayed.

U.S. Pat. No. 7,746,360 discloses a method and apparatus for bringing atarget region and a correlated visualization together, wherein if thetarget region is changed, the magnified visualization may be modified aswell.

U.S. Pat. No. 7,760,187 discloses a method wherein a region around anarea where a user touches the display (touch interaction) may beenlarged.

U.S. Pat. No. 7,889,212 discloses a method and apparatus for enablingselected portions of visual information displayed on a screen to bemagnified using a virtual loupe.

SUMMARY OF THE INVENTION

Existing limitations associated with the foregoing, as well as otherlimitations, can be overcome by methods for dynamically adjusting a 3Dmodel view and/or orientation to generate a 3D model view and/ororientation that is adapted to a surface of the model being worked on.“Dynamic adjustment” of the 3D model may be hereinafter used to meanautomatic, continuous and/or intermittent adjustment of aview/visualization/rendering and/or orientation of the 3D model. Themodel may be any model (for example a 3D model, a 2D model, a stack of2D models, etc.) and preferably, a 3D model of a tooth or teeth ordental anatomy. For example, 3D model data of a 3D model of teeth mayexist in memory and (i) parts of the 3D model data may be leftunrendered and/or may be rendered in such a way that the 3D model orparts of the 3D model on a display may appear altered (e.g deleted, madetransparent or the like) such that a user may have an optimized viewand/or (ii) an orientation of the 3D model may be changed such that auser may have a preferably unobstructed view of the 3D model surfacewherein the obstruction may be detected by. For example, determining ifa topmost triangle of a tringle mesh of the 3D model in a view direction(said topmost triangle being determined by a buffer containing zcoordinates of the 3D model closest to a scanner) is part of a surfacebeing engaged by a user. If said topmost is not part of the surfacebeing engaged, it may be concluded that the surface being engaged isobstructed and the 3D model may be re-oriented.

In an aspect herein, the present invention may provide a method fordynamically adjusting a 3D model, the method comprising the steps of:identifying a 3D model engagement process: obtaining a localized area ofthe 3D model proximate to a location of the 3D model being worked on;and adjusting the 3D model based on the obtained localized area and theengagement process.

In another aspect herein, any of the aspects of the present inventionmay be further characterized by one or any combination of the followingfeatures: (i) obtaining an orientation of the 3D model based on thelocalized area; and obtaining a view direction; wherein the adjusting isfurther based on the obtained orientation and view direction: (ii)adjusting is performed such that the orientation and view directioncoincide; (iii) wherein the adjusting is continuous and/or intermittent;(iv) wherein the adjusting includes (a) rotation of the 3D model, (b)translation of the 3D model, (c) magnification of the 3D model or a partof the model, (d) making the 3D model or a part of the model transparent(e) coloring the 3D model or a part of the model; (v) wherein theorientation is based on one or more surface normals or an average ofsurface normal of the localized area; (vi) wherein the 3D model is adental model: (vii) wherein the engagement process includesdeforming/modifying a surface of the 3D model (viii) wherein theadjusting is further based on a predetermined threshold value; (ix) thepredetermined threshold value is an angle formed by (a) one or moresurface normals or an average of surface normals of the localized areaand (b) a view direction; (x) wherein the adjusting includes a contextawareness step wherein an extent of the adjusting, is based onproperties of the 3D model; (xi) wherein the properties include toothstructure and/or tooth shape. (xii) wherein the engagement process is acontinuous movement of an operator; or (xiii) any combination thereof.

In another aspect, a system may be provided for dynamically adjusting a3D model, the system comprising: at least one processor configured to:identify a 3D model engagement process: obtain a localized area of the3D model proximate to a location of the 3D model being worked on; andadjust the 3D model based on the obtained localized area and theengagement process.

In another aspect, any of the aspects of the system may be furthercharacterized by one or any combination of the following features: (i)the processor further configured to; obtain an orientation of the 3Dmodel based on the localized area: and obtain a view direction; whereinthe adjusting is further based on the obtained orientation and viewdirection; (ii) wherein the adjusting is performed such that theorientation and view direction coincide; (iii) wherein the adjusting iscontinuous and/or intermittent; (iv) wherein the adjusting includes (a)rotation of the 3D model, (b) translation of the 3D model, (c)magnification of the 3D model or a part of the model, (d) making the 3Dmodel or a part of the model transparent (e) coloring the 3D model or apart of the model; (v) wherein the orientation is based on one or moresurface normals or an average of surface normals of the localized area;(vi) wherein the 3D model is a dental model; (vii) wherein theengagement process includes deforming a surface of the 3D model; (viii)wherein the adjusting is further based on a predetermined thresholdvalue; (ix) wherein the predetermined threshold value is an angle formedby (i) one or more surface normals or an average of surface normals ofthe localized area and (ii) a view direction, (x) wherein the adjustingincludes a context awareness step wherein an extent of the adjusting, isbased on properties of the 3D model; or (xi) any combination thereof.

Dental treatment planning may include designing steps such as drawingmarginal lines around tooth and tooth stumps, designing inlays, onlaysand partial crowns, implant planning etc. The design steps may beconducted on a display such as the display of a dental CAD/CAM machine.Methods and systems described herein may provide a means for reducingthe number of interactions a user needs to complete a treatment planningprocedure thereby improving efficiency and reducing treatment planningcompletion time for example, a method for drawing objects (e.g. lines,circles, curves, irregular shapes or the like) on a 3D model duringtreatment planning and automatically adjusting a view/orientation of the3D model based on a localized area of the 3D model proximate to alocation of the drawing. This may involve, for example, automatically,continuously and/or intermittently changing an orientation of athree-dimensional (3D) model such as a 3D tooth model, with respect to aview direction (in an embodiment herein being a direction perpendicularto a display plane), when a user interactively engages the model. Theorientation may be based on for example one or more surface normals oran average of surface normals. Herein a view angle may be for example anangle between a surface normal of the 3D model and a view direction ofthe user. The user may therefore be afforded an unobstructed view of asurface of the model being worked on, thus reducing the number of timesthe user manually changes views of the model to reveal an obstructedsurface being worked on. In an embodiment herein, a size of thelocalized area may range from for example a single point to the surfaceof a side of a tooth.

Further features and advantages, as well as the structure and operationof various embodiments herein, are described in detail below withreference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Example embodiments will become more fully understood from the detaileddescription given herein below and the accompanying drawings, whereinlike elements are represented by like reference characters, which aregiven by way of illustration only and thus are not limitative of theexample embodiments herein and wherein:

FIG. 1 is a diagram illustrating a system according to an embodiment ofthe present invention;

FIG. 2 illustrates a block diagram of an exemplary computer systemaccording to an embodiment of the present invention;

FIG. 3 is a diagram showing a 3D model and tool an embodiment of thepresent invention:

FIG. 4 illustrates a 3D model having a marginal line drawn around thebase of a tooth (e.g., preparation/stump);

FIG. 5 illustrates the 3D model of FIG. 4 having a longer marginal linearound the base of a tooth and a corresponding adjustment of theorientation of the 3D model;

FIG. 6a illustrates a view direction with respect to a surface normal;

FIG. 6b illustrates another view direction with respect to a surfacenormal;

FIG. 7 illustrates an embodiment of a method according to the presentinvention; and

FIG. 8 illustrates another embodiment of a method according to thepresent invention.

Different ones of the Figures may have at least some reference numeralsthat may be the same in order to identify the same components, althougha detailed description of each such component may not be provided belowwith respect to each Figure.

DETAILED DESCRIPTION OF THE INVENTION

In accordance with example aspects described herein, a method and systemmay be provided for manipulating a 3D model during treatment planningand automatically adjusting the 3D model view/orientation based on alocalized area of the 3D model proximate to a location of saidmanipulation.

The present invention may provide a method for deforming/modifying (suchas altering a shape) a 3D model surface and automatically adjusting anorientation of the 3D model 26 (FIG. 3) based on a localized area of the3D model 26 proximate to a location of a deformation.Drawing/deformation/interaction tools such as cusp and fissure tools maybe employed for drawing on or deforming the 3D model surface prior toadjusting the orientation of the 3D model 26. A threshold value may beused wherein the 3D model view and/or orientation are adjusted after thethreshold value is reached. In yet another embodiment herein, thethreshold value may include, for example, an angle/angles formed by oneor more surface normals 62 (FIG. 6a ) and a view direction 61. Herein,the one or more surface normals may be dynamically updated as thelocalized area changes. The adjustment of a view/orientation of the 3Dmodel 26 or a part of the 3D model may include rotation of the model,translation of the model, magnification of the model, coloring the modeland/or making the model transparent. The adjustment of theview/orientation of the model may include a context awareness step (stepinvolving the ability to gather information about the dental anatomy atany given time and adapt accordingly) wherein an extent of theadjustment, (including for example speed of rotation/translation andmagnification factor), may be dependent on knowledge of properties ofthe 3D model 26 or properties of part of the 3D model (such as toothstructure and shape).

System for Modeling and Visualizing a Dental Solution

FIG. 1 illustrates a block diagram of a system 10 comprising a userinterface 12 such as graphical user interface, tangible user interface,touchscreen interface 21, gesture driven interface or otherwise userinterface that may be used for displaying a model such as a 3D model 26of a dental anatomy as shown in FIG. 3, and which may be constructed andoperated in accordance with at least one example embodiment herein. Theuser interface 12 may be connected to or form part of a computer system11. A modeling device 14 having a modeling system 16 may be electricallyconnected to a processor 13. The modeling device 14 and/or modelingsystem 16 may form at least part of any of the devices, components,and/or systems discussed herein (for example, as a control circuitry ofthe user interface 12 to determine and/or translate user input or usergestures (such as touchscreen gestures or touchless gestures), as ahand-held controller to detect user movements/gestures and/or as memoryor visualization instructions stored in memory for performingvisualization steps discussed hereinafter). The processor 13 may be inelectrical communication with other components of the computer system 11or may be part of a different computer system. To execute a procedure,the processor 13 may load appropriate instructions, as stored on adevice, into memory and may then execute the loaded instructions.

The system may optionally obtain data for generating the 3D model froman external device 15 such as an optical, laser or x-ray scanner or thesystem may generate the 3D data from another plurality of 3D data.

The system may include one or more input devices 18, 19, for example, amouse, a keyboard, a stylus, a trackball, or a natural input such as afinger for drawing on or deforming the 3D model surface.

In one specific embodiment herein, the user interface 12 is preferably atouchscreen interface 21 and the input device 18, 19, is the finger.Herein, the modeling system 16 may detect different operator 25 (finger)gestures on the touchscreen interface 21 and map said detected gesturesto corresponding actions as discussed hereinafter. Dynamic adjustment ofthe 3D model view/orientation may be performed by the processor 13 intandem with modeling system 16 wherein engagement of the user interfaceby the operator 25 to draw on or modify a surface 32 of the 3D model 26may result in initiating the dynamic adjustment to adjust the 3D modelaccording to predetermined criteria as the engagement process (drawingon, deformation of, scaling of, or the like of the 3D model by a user)progresses. Herein manual adjustment (as opposed to dynamic adjustment)of the perspective of the model by the user may be reduced or eliminatedsuch that the user can focus on the design. Instructions for saiddynamic adjustment may be controlled by the modeling system 16 and mayinclude but may not be limited to (i) rotation of the model, (ii)translation of the model, (iii) magnification of the model or a part ofthe model, (iv) making the model or a part of the model transparent (v)coloring the model or a part of the model, (vi) etc. Herein, the 3Dmodel data may exist in memory and a rendering of the 3D model data asseen on the interface 12 may be changed. Said dynamic adjustment may beperformed according to criteria which may optionally be set by the userand may include but may not be limited to (i) speed ofrotation/translation and (ii) magnification factor of a magnification ofa surface 32 of the 3D model. Moreover, the criteria for said dynamicadjustment may optionally be based on context awareness of the 3D model,for example it may be based on properties of the 3D model including butnot limited to (i) position/size of adjacent teeth (ii) strength ofteeth, (iii) number of restorations needed in the design, (iv)complexity of the design of the restoration, aligner and/or (v) the likeetc. It will be appreciated by a person of ordinary skill in the artthat similar systems may be achieved for other user interfaces and theircorresponding operators 25 according to systems described herein.

Computer System for Modeling and Visualizing a Dental Solution

Having described a system 10 for dynamically adjusting aview/orientation of the 3D model based on a localized area of the 3Dmodel, reference will now be made to FIG. 2, which shows a block diagramof a computer system 100 that may be employed in accordance with atleast some of the example embodiments described herein. Although variousembodiments may be described herein in terms of this exemplary computersystem 100, after reading this description, it may become apparent to aperson skilled in the relevant art(s) how to implement the disclosureusing other computer systems and/or architectures.

In one example embodiment herein, at least some components of thecomputer system 100 may form or be included in the computer system 11 ofFIG. 1. The computer system 100 may include at least one computerprocessor 122 and at least one modeling device 150, modeling system 152,user interface 126 and input unit 130 which may form at least part ofany of the devices, components, and/or systems discussed herein and/ormay respectively form or be included in the processor 13, modelingdevice 14, modeling system 16, user interface 12 and input device 18, 19of FIG. 1. The computer processor 122 may include, for example, acentral processing unit, a multiple processing unit, anapplication-specific integrated circuit (“ASIC”), a field programmablegate array (“FPGA”), or the like. The processor 122 may be connected toa communication infrastructure 124 (e.g., a communications bus, or anetwork). In an embodiment herein, the processor 122 may receive anindication that a design is in progress or the 3D model 26 is beingengaged and may obtain instructions concerning the dynamic adjustment ofthe 3D model view/orientation from a memory of the modeling system 152and/or a from one or more storage units of the computer system 100. Theprocessor 122 may then load the instructions and execute the loadedinstructions. This dynamic adjustment of the 3D model view/orientationmay then be rendered on the display unit 128.

The user interface (or other output interface) 126 may forward videographics, text, and other data from the communication infrastructure 124(or from a frame buffer (not shown)) for display on the display unit 128(which, in one example embodiment, may form or be included in thedisplay unit 128 of FIG. 1). For example, the user interface 126 mayinclude a video card with a graphics processing unit.

The computer system 100 may also include an input unit 130 that may beused by a user of the computer system 100 to send information to thecomputer processor 122. In one exemplary embodiment herein, the inputunit 130 is a finger or stylus to be used on a touchscreen interface 21.The input unit 130 may alternatively be a trackball or other inputdevice such as a keyboard or stylus or gesture recognition device. Inone example, the display unit 128, the input unit 130, and the computerprocessor 122 may collectively form a user interface 126.

One or more steps of generating the dynamic adjustments may be stored ona non-transitory storage device in the form of computer-readable programinstructions. To execute a procedure, the processor 122 loads theappropriate instructions, as stored on a storage device, into memory andthen executes the loaded instructions.

The computer system 100 of FIG. 2 may further comprise a main memory132, which may be a random access memory (“RAM”) and also may include asecondary memory 134. The secondary memory 134 may include, for example,a hard disk drive 136 and/or a removable-storage drive 138 (e.g., afloppy disk drive, a magnetic tape drive, an optical disk drive, a flashmemory drive, and the like). The removable-storage drive 138 may readfrom and/or write to a removable storage unit 140 in a well-knownmanner. The removable storage unit 140 may be, for example, a floppydisk, a magnetic tape, an optical disk, a flash memory device, and thelike, which may be written to and read from by the removable-storagedrive 138. The removable storage unit 140 may include a non-transitorycomputer-readable storage medium storing computer-executable softwareinstructions and/or data.

In further alternative embodiments, the secondary memory 134 may includeother computer-readable media storing computer-executable programs orother instructions to be loaded into the computer system 100. Suchdevices may include a removable storage unit 144 and an interface 142(e.g., a program cartridge and a cartridge interface): a removablememory chip (e.g., an erasable programmable read-only memory (“EPROM”)or a programmable read-only memory (“PROM”)) and an associated memorysocket; and other removable storage units 144 and interfaces 142 thatallow software and data to be transferred from die removable storageunit 144 to other parts of the computer system 100.

The computer system 100 also may include a communications interface 146that enables software and data to be transferred between the computersystem 100 and external devices. Such an interface may include a modem,a network interface (e.g., an Ethernet card or an IEEE 802.11 wirelessLAN interface), a communications port (e.g., a Universal Serial Bus(“USB”) port or a FireWire® port), a Personal Computer Memory CardInternational Association (“PCMCIA”) interface, Bluetooth®, and thelike. Software and data transferred via the communications interface 146may be in the form of signals, which may be electronic, electromagnetic,optical or another type of signal that may be capable of beingtransmitted and/or received by the communications interface 146. Signalsmay be provided to the communications interface 146 via a communicationspath 148 (e.g., a channel). The communications path 148 carries signalsand may be implemented using wire or cable, fiber optics, a telephoneline, a cellular link, a radio-frequency (“RF”) link, or the like. Thecommunications interface 146 may be used to transfer software or data orother information between the computer system 100 and a remote server orcloud-based storage (not shown).

One or more computer programs or computer control logic may be stored inthe main memory 132 and/or the secondary memory 134. The computerprograms may also be received via the communications interface 146. Thecomputer programs may include computer-executable instructions which,when executed by the computer processor 122, cause the computer system100 to perform the methods as described hereinafter. Accordingly, thecomputer programs may control the computer system 100 and othercomponents of the systems for modeling and visualizing a dentalsolution.

In another embodiment, the software may be stored in a non-transitorycomputer-readable storage medium and loaded into the main memory 132and/or the secondary memory 134 of the computer system 100 using theremovable-storage drive 138, the hard disk drive 136, and/or thecommunications interface 146. Control logic (software), when executed bythe processor 122, causes the computer system 100, and more generallythe system for modeling and visualizing a dental solution, to performall or some of the some of the methods described herein.

Implementation of such other hardware arrangement so as to perform thefunctions described herein will be apparent to persons skilled in therelevant art(s) in view of this description.

Method for Modeling and Visualizing a Dental Solution.

Having described the computer system 100 of FIG. 2, the methods formodeling and visualizing a dental solution 101 will now be furtherdescribed in conjunction with FIGS. 7-8 which show methods ofdynamically adjusting a view and/or orientation of a model based on alocalized area of the model during treatment planning.

The invention may include adjusting the 3D model when a user is engagingand/or interacting with (such as moving, placing objects on, drawing on,etc.) a surface 32 or location 31 (such as a point on the 3D modelresulting from a click, touch, or otherwise interaction between the userthrough an input device (or otherwise) and the 3D model) of the 3D model26. During such interaction, the 3D model 26 may be automaticallyadjusted with little to no input from the user in a way that may allowthe user to have an unobstructed view of the surface 32 being engaged.This adjustment may be based on a localized area of the 3D modelproximate to the engagement location/point 31. The adjustment may alsopreferably be continuous (and optionally automatic) such that the usermay have a continuously unobstructed view of any surface 32 beingengaged by the user as the design progresses. Herein, the user may have,for example, an orthogonal view of the surface 32 being engaged.Alternatively any parts of the 3D model that may be blocking the surface32 currently being engaged may be removed or made transparent through 3Drendering methods. Therefore the user may continue designing on and/ormodifying the surface 32 without having to stop to manually correct theorientation of the 3D model, because for example, objects moved alongthe surface 32 shifted out of sight. As the 3D model 26 is dynamicallyadjusted, the user may save time and the design process may be moreefficient.

In an exemplary embodiment of the invention herein, a click or touch ofa area on the 3D model may define a location/point 31 or a surface 32wherein an orientation of the 3D model with respect to a user direction(for example an angle α, as shown in FIG. 6a , between a surface normal62 of the location/point 31 or an average of surface normals (not shown)of points in the surface 32 with respect to view direction 61) may becalculated. In an exemplary embodiment, the calculation may be achievedby (i) defining a region, for example in mm² (ii) determining trianglesof a triangle mesh in the defined region (iii) computing normal of thesetriangles (iv) defining a common vector by adding all normal vectors,each multiplied by a surface size of their corresponding triangles (v)and normalizing the common vector (eg. by giving it a length “1”). Theorientation of the surface 32 or location/point 31 may then be adaptedby for example rotating the 3D object so that the calculated surfacenormal or average of surface normals fits a direction of the viewer asshown in FIG. 6 b.

In another exemplary embodiment herein, drawings such as lines, forexample, a first marginal line 22 may be drawn on the 3D model 26.During the drawing of the first marginal line 22 the orientation/viewdirection on the 3D model 26 may continuously follow the surface normalof the current interaction location/point 31. Herein, during thisadaptation/dynamic adjustment of the orientation, one or more surfaces32 of the current interaction point 31 may be magnified/demagnified todisplay a magnified/demagnified surface 24 of the one or more surfaces32. In a similar embodiment, dynamic adjustment of the 3D modelview/orientation may be configured to correspond to context awareness ofthe surface 32 being engaged. For example, anatomically significantareas of the 3D model (such as large curvatures, edges, breaks, holes,teeth cusps, fissures, equator or the like) may be detected through forexample mathematical means or through object recognition means and anautomatic zoom in/out or view direction adaption of the surface 32 maybe initiated, preferably automatically. Moreover the user may slow downor accelerate his design or movement speed on the surface 32 and theautomatic zoom in/out or view direction adaption of the surface 32 maybe initiated for that surface 32. Furthermore a size of the zoomed areamay be adapted to correspond to a resolution or size of the userinterface 12. This may guarantee a “real world” dimension of themagnification on the interface 12. Yet still, the magnified/demagnifiedsurface 24 may be steered and/or controlled by the user during thedesign. Herein gestures such as finger gestures on a touchscreen 21 maybe used wherein for example a size of the magnified/demagnified surface24 or zoom factor may be changed. In a first example, a thumb may beon/outside a border of the magnified/demagnified surface 24 while a 2ndfinger may enlarge/shrink the size of the border (area being magnified).In a second example both fingers may be placed within a border of themagnified area to control the zoom factor. In a third example, twofingers may concurrently change the size of the magnifier and the zoomfactor. Herein, the user may keep one finger on the touchscreen 21 (e.g.while drawing a line and the magnifier is active). When a second fingeris placed on the touchscreen 21, the drawing may be paused to allow themagnifier and its zoom factor to be controlled. Implementation of thesame or other gestures in the same or other user interfaces 12 so as toperform the same or similar functions described herein will be apparentto persons or ordinary skill in the relevant art(s) in view of tisdescription.

In another exemplary embodiment herein, drawings such as lines, forexample, a second marginal line 23 may be drawn on the 3D model 26.During the drawing of the second marginal line 23 the orientation/viewdirection on the 3D model 26 may continuously follow the surface normalof the current interaction point. Herein, during this adaptation/dynamicadjustment of the orientation, one or more surfaces 28 of the 3D model26 may block a view of the surface being engaged. The one or moresurfaces 28 preventing the direct view may then be for example, hidden,clipped or rendered transparent as shown in FIG. 3. In a furtherembodiment, the method may be applicable when using interaction tools 27such as 3D CAD interaction elements including elements that allow (i)positioning of markers on the 3D model 26, (ii) deforming of a surfaceof the 3D model 26 (iii) movement or using of interaction elements whichhave a 3D correlation in position with respect to the 3D model's surfaceand whose correlation is changed by the user. Herein the model may beautomatically rotated or zoomed, if the user moves the interactionelements over/along a surface of the 3D model 26, so that they are notmoved to a side or backside of the 3D model 26 with respect to the viewdirection of the user. For example, FIG. 4 shows a 3D model 26 a havinga marginal line 22 a drawn around the base of a tooth being prepared 42.As a user extends the marginal line 22 a into an extended marginal line22 b, the orientation of the 3D model 26 a may be changed from a firstorientation as shown in FIG. 4 to a second orientation as shown in FIG.5 such that a portion 44 of the dental anatomy is visible in or order todraw the extended marginal line 22 b.

In another embodiment of the method, the adjustment of the view and/ororientation of the 3D model 26 may be done partially. Herein theadjustment may be done after a threshold is reached. The threshold mayoptionally be predetermined. The threshold may for example be based on(i) the magnitude of an angle between the surface normal and the viewdirection and/or (ii) size of a drawing (e.g. length of a drawn line)and/or (iii) accumulated covered curvature of the surface by a drawing.Similarly, a latency may be adopted such that dynamic adjustment occursafter the latency (such as an observed time period) is observed beforeadjusting in order to reduce or eliminate a “hopping” or erratic changein views. In an embodiment, a threshold value for the magnitude of theangle between the surface normal and the view direction may be, forexample, an angle greater than 3 (for example and angle greater than 5).In another embodiment, a threshold value for a length of a drawn line(eg. from a starting position to an current position) may be a lengthgreater than 3 mm (for example a length greater than 5 mm). In anotherembodiment, a threshold value for the length of an accumulated curvature(eg. the curvature between a starting position and a current position)of a drawing/curve may be a length greater than 3 mm (eg. a lengthgreater than 5 mm). In another embodiment, a threshold value for alatency value (eg. a time from the start of a drawing to a current time)may be a time greater than 0.5 s (eg. a time greater than 1 s).

In yet another embodiment, dynamic adjustment may be initiated by auser. Herein the user may manually adjust the 3D model 26 until acommand is given by the user to initiate the automatic, continuousand/or intermittent adjustment based on for example a threshold value ofa drawing. In another embodiment, a drawing process may be paused as thedynamic adjustment occurs such that the drawing may not be completedoff-course. Moreover, the dynamic adjustment may optionally be initiatedafter an ongoing drawing reaches a surface that is out of the user'sview. In yet another exemplary embodiment a change of the orientation ofthe 3D model may be based on a pivot point wherein the pivot point isthe last or current point being engaged by the user and wherein the 3Dmodel is turned on that pivot point.

In another embodiment of the method, to avoid unsteady or flickeringmovements of the 3D model during dynamic adjustment, a thresholdcompensation smoothing calculation may be useful for the adjustment ofthe view/orientation wherein the threshold compensation smoothingcalculation may relates to an accumulated curvature and/or distance onthe 3D surface by a user drawing, so that the adjustments during theinteraction with the 3D model may not lead to having the view tumbleunnecessarily.

FIGS. 7-8 show example methods described herein comprising rendering a3D model on a display in Step S10. A processor 13, 122 in tandem with amodeling system 16 may detect a user engagement or interaction processand may adjust the 3D model based on a detected localized area and theengagement process. For example, the processor 13, 122 may detect asingle click or continuous move of an operator 25. The 3D model 26 maypreferably remain unadjusted if processor 13, 122 detects a singleclick. If the processor 13, 122 detects a continuous movement of anoperator 25, the processor may adjust the 3D model by for examplerotating the 3D model as shown in Step S13. The adjustment may be basedon an angle α between a view direction of the user 61 and a surfacenormal of the position of the operator 25, (such as a finger position ona surface of the 3D model 26 using a touchscreen display) exceeding apredetermined value x, Step S12. In another embodiment as shown in FIG.8, the adjusting (e.g. rotation) of the 3D model may be continuous oncethe angle α exceeds the predetermined value x as in Step S14.

In view of the foregoing description, it may be appreciated that theexample embodiments described herein provide a method and system fordynamically adjusting a view/orientation of the 3D model 26 based on alocalized area of said 3D model 26. The method and system may be usefulfor reducing the number of times a user manually adjusts aview/orientation of the model during treatment planning.

Unless otherwise defined, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Although methods and materialssimilar to or equivalent to those described herein may be used in thepractice or testing of the disclosure, suitable methods and materialsare described above. All publications, patent applications, patents, andother references mentioned herein are incorporated by reference in theirentirety to the extent allowed by applicable law and regulations. Thedisclosure may be embodied in other specific forms without departingfrom the spirit or essential attributes thereof and it may therefore bedesired that the present embodiment be considered in all respects asillustrative and not restrictive. Any headings utilized within thedescription are for convenience only and have no legal or limitingeffect.

The invention claimed is:
 1. A method for dynamically adjusting a 3Dmodel, the method comprising the steps of: engaging a 3D model on ascreen, using a 3D model engagement process; identifying the 3D modelengagement process; obtaining a localized area of the 3D model proximateto a location of the 3D model being engaged by defining a region of the3D model proximate to the location of the 3D model being engaged, thelocalized area of the 3D model and the location of the 3D model beingengaged are on the same screen and the 3D model is a dental 3D model;computing an orientation of the obtained localized area or location ofthe 3D model being engaged; and adjusting the 3D model based on theorientation of the obtained localized area or location of the 3D modelbeing engaged and the engagement process by making the 3D model or apart of the model transparent, and/or coloring the 3D model or a part ofthe model.
 2. The method of claim 1, wherein the adjusting is based on acontext awareness of the 3D model.
 3. The method of claim 2, wherein theadjusting is based on a position/size of an adjacent tooth.
 4. Themethod of claim 2, wherein the adjusting is based on a strength of atooth.
 5. The method of claim 2, wherein the adjusting is based on anumber of restorations needed in a design.
 6. The method of claim 2,wherein the adjusting is based on a complexity of a design of arestoration.
 7. The method of claim 1, wherein the adjusting isautomatic, continuous and/or intermittent.
 8. The method according toclaim 1, wherein the engagement process includes deforming/modifying asurface of the 3D model.
 9. A system for dynamically adjusting a 3Dmodel, the system comprising at least one processor configured toperform the steps of: identifying a 3D model engagement process;obtaining a localized area of the 3D model proximate to a location ofthe 3D model being engaged by defining a region of the 3D modelproximate to the location of the 3D model being engaged, the localizedarea of the 3D model and the location of the 3D model being engaged areon a same screen and the 3D model is a dental 3D model; computing anorientation of the obtained localized area or location of the 3D modelbeing engaged; and adjusting the 3D model based on the orientation ofthe obtained localized area or location of the 3D model being engagedand the engagement process by making the 3D model or a part of the modeltransparent, and/or coloring the 3D model or a part of the model. 10.The system of claim 9, wherein the processor is further configured toadjust the 3D model based on a context awareness of the 3D model. 11.The system of claim 10, wherein the processor is further configured toadjust the 3D model based on a position/size of an adjacent tooth. 12.The system of claim 10, wherein the processor is further configured toadjust the 3D model based on a strength of a tooth.
 13. The system ofclaim 10, wherein the processor is further configured adjust the 3Dmodel based on a number of restorations needed in a design.
 14. Thesystem of claim 10, wherein the processor is further configured toadjust the 3D model based on a complexity of a design of a restoration.15. The system of claim 9, wherein the adjusting is automatic,continuous and/or intermittent.
 16. The system according to claim 9,wherein the engagement process includes deforming/modifying a surface ofthe 3D model.
 17. A non-transitory computer-readable storage mediumstoring a program Which, when executed by a computer system, causes thecomputer system to perform a procedure comprising: identifying a 3Dmodel engagement process; obtaining a localized area of the 3D modelproximate to a location of the 3D model being engaged by defining aregion of the 3D model proximate to the location of the 3D model beingengaged, the localized area of the 3D model and the location of the 3Dmodel being engaged are on a same screen and the 3D model is a dental 3Dmodel; computing an orientation of the obtained localized area orlocation of the 3D model being engaged and; and adjusting the 3D modelbased on the orientation of the obtained localized area or location ofthe 3D model being engaged and the engagement process by making the 3Dmodel or a part of the model transparent and/or coloring the 3D model ora part of the model.